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Image segmentation generally concerns selection and/or separating a selected 
part of a dataset. Notably such a dataset represents image information of an imaged object 
and the selected part relates to a specific part of the image. The dataset is in general a multi- 
dimensional dataset that assigns data-values to positions in a multi-dimensional geometric 
space. In particular, such datasets can be two-dimensional or three-dimensional images where 
the data- values are pixel-values, such as brightness values, grey values or colour values, 
assigned to positions in a two-dimensional plane or a three-dimensional volume. 



10 The invention pertains to a method of segmenting a selected region from a 

multi-dimensional dataset comprising the steps of 

- setting-up a shape model representing the general outline of the selected region 

- setting-up an adaptive mesh representing an approximate contour of the selected region 

- which adaptive mesh is 

15 - initialised on the basis of the shape model . 



Such a method of segmenting a selected region from an three-dimensional 
dataset is known from the paper 'An efficient 3D deformable model with a self-optimising 
20 mesh ' by AJ. Bulpitt and N.E. Efiford in Image and Vision Computing 14(1996)573-580. 

The known method operates on a multi-dimensional dataset in the form of a 
three-dimensional image. The known method employs a triangular mesh to represent a 
surface of the selected region. A so-called distance transform is used to initialise the adaptive 
mesh and when the mesh is close to its final solution an image grey level gradient is used to 
25 drive the deformation of the mesh. 
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An object of the invention is to provide a method of segmenting a selected 
region from a dataset which is computationally fast, more robust and more accurate than the ' 
conventional method. 

This object is achieved by the method of segmenting a selected region from a 
5 dataset wherein according to the invention the adaptive mesh is deformed depending on the 
shape model and on feature information of the selected region. 

f 

The selection of the selected region may be effected by the initialisation of the 
10 segmentation. In particular the initialisation is done on the basis of the shape model. Such a 
shape model represents the selected region in terms of a parameterisation of a priori shape 
information. This a priori shape information and the positioning of the shape model in the 
multi-dimensional dataset determines the region that is being selected. In particular, the shpae 
model is an active shape model that allows for adaptation of its translation and orientation 

15 and pose, but also of shape parameters representing smaller details of the shape of the region 
that is represented by the active shape model. The adaptive mesh represents an approximate 
contour. The approximate contour represents an estimate of the boundary of the selected 
region relative to the remainder of the multi-dimensional dataset. The contour is in general a 
hypersurface in the multi-dimensional dataset, such as a surface or a curve. The contour has a 

20 dimension that is lower than the dimensionality of the multi-dimensional dataset As the 
adaptive mesh is deformed, the approximate contour more accurately approaches the 
boundary of the selected region. According to the invention the adaptive mesh is being 
deformed on the basis of feature information together with information from the shape 
model. The feature information for example concerns image features such as edges and 

25 surfaces of about equal data- values. Such feature information is preferably represented by 
local gradients in the dataset, i.e. local relative changes of the data- values. Strong gradients 
indicate the presence of boundaries and the direction of the gradients is locally 
perpendicularly to the boundary. The shape model is being employed during the successive 
steps of adaptation of the mesh. For example, the adaptive mesh is deformed on the basis of 

30 an internal energy and according to the invention the internal energy is defined in dependence 
on the shape model. Thus, information provided by the shape model is more effectively 
employed to drive the adaptive mesh to the actual boundary of the selected region. Notably, 
image features in the multi-dimensional dataset which would drive the adaptive mesh to false 
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boundaries, so-called false attractors, are substantially avoided, while a particular close 
initialisation is not required. 

These and other aspects of the invention will be further elaborated with 
references to the preferred implementations and embodiments as defined in the dependent 
5 Claims. 

Preferably, the shape model is updated upon deformation of the adaptive 
mesh. In particular the parameters of the shape model such as the position, scale and pose are 
adapted as the boundary is approached by the adaptive mesh. 

In a further preferred implementation of the invention on or several local 

10 surface patches of the selected region are detected. An individual local surface patch is a 
small area tangent to the boundary of the selected region. In practice the local surface patch 
is taken as a small region transverse to a local gradient in the neighbourhood of the current 
adaptive mesh. During mesh adaptation, boundaries are detected, which are in agreement 
with the adaptive mesh and no care is taken whether they belong to the object, which should 

15 be segmented. Normally, some boundaries are correctly detected and others are misleading in 
the first few iterations of mesh adaptation. As the correct boundaries are more consistent with 
the model than the misleading boundaries, the fraction of correctly detected boundary points 
increases during mesh adaptation. 

Subsequently, an individual vertex of the adaptive mesh is moved in the 

20 direction transverse to the local surface patch at issue so as to deform the mesh. In this way it 
is avoided that the mesh is deformed in the direction of 'false attractors'. 

Preferably, the deformation of the mesh is carried-out in that individual 
vertices of the adaptive mesh are moved toward to the surface patch dependent on the angle 
between the local normal to the adaptive mesh and the local normal to the surface patch. 

25 Usually, the individual vertex is moved towards the surface patch that is closest to the vertex 
at issue. In this way the adaptive mesh is deformed while adequately taking available 
information on image features into account. Notably, the image features as represented by the 
orientation of the local surface patch is most relevant for deformation of the adaptive mesh 
while avoiding influence of so-called false attractors. Particularly advantageous results are 

30 obtained when the vertices are moved essentially perpendicularly towards the local surfaces 
patches. 

Further, advantageously, gradients in the dataset having a magnitude smaller 
than a pre-set threshold value are discarded for forming the surface patches and deformation 
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of the adaptive mesh. Thus, deviations in the adaptive mesh deformation due to false 
attractors is further avoided. 

A preferably implementation of the adaptive mesh deformation is made on the 
basis of so-called energy functions. The term 'energy 1 in this respect is not related to a 
physical energy, but is used because the concept of a deformable adaptive mesh has strong 
resemblance with a string of masses (the vertices) interconnected by springs (the bonds) 
moving in under the influence of a force field and striving towards a stable situation of 
minimum energy. In this resemblance, the external energy pertains to the external force field 
applied to the vertices and the internal energy pertains to the mutual interaction of between 
the vertices. Notably, the internal energy relates to the shape of the adaptive mesh. 
According to the invention, preferably the relative weight of the external energy relative to 
the internal energy is controlled by a parameter that may be adjusted by the user. This 
provides additional flexibility in implementing the deformation of the adaptive mesh in that 
the relative influences of the force fields and the shape of the adaptive mesh can be adapted. 
Preferably, the external energy is made dependent on feature information of the selected 
region relative to the actual configuration of the adaptive mesh. Thus, it is achieved that the 
external forces drive the adaptive mesh towards image features, such as strong boundaries, of 
the selected region. In order to control the shape of the adaptive mesh to resemble to a certain 
degree the shape of the selected region, advantageously the internal energy is dependent on 
the shape model. Thus it is avoided that the external forces cause the adaptive mesh to 
strongly deviate from the rough shape of the selected region. The co-operation of the internal 
energy and the external energy leads to fast and reliable convergence of the adaptive mesh 
accurately to the actual boundary of the selected region. As the shape model is updated upon 
deformation of the adaptive mesh, the co-operation of the internal energy and the external 
energy is further improved in that the most updated information is employed and deviations 
due to false attractors is effectively avoided. 

The invention also relates to a dataprocessor for segmenting a selected region 
from a multi-dimensional dataset. The dataprocessor according to the invention is defined in 
Claim 9. The dataprocssor according to the invention is able to carry-out the method 
according to the invention and notably avoid deviations due to false attractors. 

The invention also relates to a computer programme including instructions for 
segmenting a selected region from a multi-dimensional dataset. The computer programme 
according to the invention is defined in Claim 10. The computer programme according to the 
invention is preferably loaded into the working memory of a dataprocessor. Thus the 
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dataprocessor is equipped to carry-out the method of the invention. The computer programme 
may be supplied on a data carrier such as a CD-rom. The computer programme may also be 
supplied over a network such as the world-wide web and can be downloaded from such a 
network into the working memory of the dataprocessor. 

These and other aspects of the invention will be elucidated with reference to 
the embodiments described hereinafter and with reference to the accompanying drawing 
wherein 



10 The Figure illustrates the deformation of the adaptive mesh as employed in the 

method of the invention. 



In an actual implementation of the method of the invention, a triangular 
15 adaptive mesh represents the deformable model. The adaptive mesh comprises N vertices 
with co-ordinates Jc, ,..Jc^ . The adaptive mesh is adapted with an iterative procedure, where 
each iteration includes the following two steps: 

1. Surface detection is carried out to detect local surface patches of the selected region; 

2. Reconfiguration of the adaptive mesh to update the mesh in that the vertices of the mesh 
20 are moved towards the local surface patches. 

The reconfiguration of the mesh is done by minimising the energy. 
E = E t + aE . 

The external energy E ext drive the adaptive mesh towards the surface patches 
25 obtained in the surface detection step. The internal energy E int restricts the flexibility of the 
adaptive mesh. The parameter a weights the relative influence of both terms. 

The individual steps are now discussed in more detail. 
Surface detection is carried out the triangle centres x, of the adaptive mesh. A search is for 

surface patches performed along the triangle's normal h l to find the point xti with the optimal 

30 combination of feature value F(x) and distance jd to the triangle centre x t : 



x f = x i +5h i arg^min jDj 2 6 2 -F{x i + y<5w, )] 
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The parameter D controls the combination of distance information with the feature value 
F(x), the parameter / defines the length of the profile searched for surface patches and the 
parameter 8 is the distance between sample points on the profile. 

By combining feature value and distance information, surface points next to 
the adaptive mesh are preferred and the deformation of the adaptive mesh becomes much 
more stable as compared to detection of the point with maximum feature value. Thus, 
deviation due to false attractors is effectively avoided or reduced. 

For example, the quantity 

KM***.-) 
n»g = ±n»g\g nax 



10 F(x) = ± 



gm** +\\g 



M 



15 



20 



25 



is used as feature value, where the sign is chosen in dependence of the brightness in the 
selected region with respect to its surroundings. For surface points with a gradient magnitude 
|gi | smaller than the threshold value g^, this feature value is essentially the gradient in the 
direction of the local normal to the adaptive mesh. If the gradient value is above the threshold 
value, the feature value represents the scalar product between the local normal to the mesh 
and the gradient direction. Thus, the feature value takes the angle between the gradient and 
the local normal to the mesh into account. 

Advantageously, the external energy has the form 



f _ 



with the weights 

w, =max[o,F(jc,)-Z)/ 2 J 2 ] 

This external energy causes the vertices of the adaptive mesh to move perpendicularly to the 
local surface patch as shown in the Figure. 

A good choice of the internal energy defined as follows. The starting point is a 
shape model represented by a mesh of triangles. The vertex co-ordinates of the mesh of the 
shape model are given by. 
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In this expression m? i ....mf* denote the vertex co-ordinates of the mean model, and m*, ...m/f 
describe the variation of the vertex co-ordinates associated with the eigenmodes (k=J, ...M) 
of the model, /?* . .pu denoting the weights of the eigenmodes. 

The internal energy is designed to maintain the distribution of vertices to some 
extent. Thus, pathological deformations due to false attractors is avoided. 
The internal energy is given as: 

2 



E «* = Z £ \x.-Xj-sR 

vertices i neighbours j I 



j=\,.M 



1 0 where s and R represent the scale and orientation of the shape model. 

The deformation of the adaptive mesh is done in two steps. In the first step 
scaling s and orientation R of the shape model with the current weights p, of the eigenmodes 
is determined with respect to the current configuration of the adaptive mesh. Point-based 
registration methods based on singular value decomposition are found to function 

15 appropriately to determine scaling and rotation. In the second step, the vertex co-ordinates xi 
and the weights /?, are updated using the scaling s and the orientation R as determined in the 
first step. The energy function E is a quadratic function with respect to these parameters. 
Minimising this energy is conveniently done by way of a conjugate gradient method, which 
is known as such from the handbook 'Practical Optimisation' (Academic Press, San Diego 

20 1981) by P.E. Gill et al. This method takes advantage of the fact that the matrix involved in 
the minimisation is sparsely occupied. 

The deformation of the adaptive mesh until convergence to the accurate 
approximation of the boundary of the selected region takes only a short time, up to about 30s, 
which makes the segmentation according to the invention quite practical even in the event 

25 that large datasets are involved. Notably, such large datasets may occur in medical diagnostic 
applications where three-dimensional datasets with high spatial resolution are employed. 
Good results are obtained especially in the segmentation of vertebrae from three-dimensional 
images of the human spine. This application is when no steps were taken quite prone to false 
attractors formed by portions of neighbouring vertebrae. 
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1 . A method of segmenting a selected region from a multi-dimensional dataset 

comprising the steps of 

- setting-up a shape model representing the general outline of the selected region 

- setting-up an adaptive mesh representing an approximate contour of the selected region 
5 - which adaptive mesh is 

- initialised on the basis of the shape model and wherein 

- the adaptive mesh is deformed depending on the shape model and on feature 
information of the selected region. 

10 2. A method of segmenting a selected region as claimed in Claim 1 , wherein the 

shape model is updated upon deformation of the adaptive mesh. 

3. A method of segmenting a selected region as claimed in Claim 1 , wherein 

- one or more local surface patches of the selected region are detected and 

15 - the mesh is deformed in dependence of the local orientation of the mesh relative to the 
local surface patch(es). 



20 - 



A method of segmenting a selected region as claimed in Claim 3, 
the adaptive mesh including vertices and links connecting individual vertices wherein 
the mesh is deformed in that individual vertices are moved towards respective surface 
patches. 



5. A method of segmenting a selected region as claimed in Claim 4, wherein 

individual vertices are moved in dependence of the angle between the local normal to the 
25 mesh and the normal to the surface patch. 



6. A method of segmenting a selected region as claimed in Claim 5 wherein 

individual vertices are moved in the direction normal to the surface patch. 



o 
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7 - A method of segmenting a selected region as claimed in Claim 1 , wherein the 

mesh adaptation is performed on the basis of optimising a value of an energy function 

- the energy function having an internal energy contribution depending on the shape model 
and 

5 - an external energy contribution depending on feature information of the selected region 
and the actual configuration of the adaptive mesh. 

8. A method of segmenting a selected region as claimed in Claim 7, wherein the 
energy function includes a weighted combination of the internal energy contribution and the 

10 external energy contribution, involving adjustable eight factors. 

9. A data processor arranged to 

- set-up a shape model representing the general outline of the selected region 

- set-up an adaptive mesh representing an approximate contour of the selected region 
15 - which adaptive mesh is 

- initialised on the basis of the shape model and 

- deform the adaptive mesh depending on the shape model and on feature information 
of the selected region. 

20 10. A computer programme including instructions to 

- set-up a shape model representing the general outline of the selected region 

- set-up an adaptive mesh representing an approximate contour of the selected region 

- which adaptive mesh is 

- initialised on the basis of the shape model and 

25 - deform the adaptive mesh depending on the shape model and on feature information 

of the selected region. 




A method of segmenting a selected region from a multi-dimensional dataset 



comprising the steps of setting-up a shape model representing the general outline of the 
selected region and setting-up an adaptive mesh. The adaptive mesh represents an 
approximate contour of the selected region. The adaptive mesh is initialised on the basis of 
5 the shape model. Further, the adaptive mesh is deformed depending on the shape model and 
on feature information of the selected region. 
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